RRSP RS 
I ae A ed 





abraham MS#47; vismusic 1 July 26, 1988 
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by 
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Dedicated to: Father Louis-Bertrand Castel, 1688-1757. 


Abstract. Two of my experimental projects in chaos and vibration theory, over these past 
fifteen years, have developed into visual musical instruments. The Macroscope Project 
flourished from 1974 to 1979, and the MIMI Project, from 1985 to the present. ‘In this 
brief memo, a personal history, their genesis is traced. 7 
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I. My walkabout in chaos. Once upon a time, I worked on a single mathematical 
problem for six years. The project culminated in two books, written simultaneously. 
When the manuscripts were done, I looked up to see what was happening in ordinary 
reality. It was Princeton University in 1967. The Sixties were happening. A full-bore 
revolution was on. A lot of people were departing ordinary reality for one-day trips. 
Like Joseph Knecht, I decided on the spur of the moment to try it out. For me, this was 
no one-day trip. When I came down, it was 1973. 


During this six-year walkabout, I moved from Princeton to the University of California at 
Santa Cruz (UCSC), toured spiritual groups and techniques, saw my family expand and 
contract, took leave to explore Europe Hippie style, searched the Himalayan foothills for 
traditional knowledge, trekked to the borders of Tibet. In Santa Cruz there were rock 
concerts with light shows. In Amsterdam there were esoteric books and video artists, and 
floors to sleep on. In India there were yogis, gurus, classical North Indian music teachers, 
Sanscrit texts in English translation, and caves to sleep in. 


Reentry was difficult. After a cooling off period in Tahoe, supported by mathematics 
applied to blackjack, I returned to my post at the University of California in January, 
1974. I tried to integrate what I had gained from my journey with my work as a 
professor. I gave seminars in vibrations, combining the Vedic concepts with European 
mathematics. My goal, inspired more by desperate need to communicate than the ideals 
of Boddhisatva or Toynbee, was to share the essential visual aspect of the vibratory fields 
I had seen in my travels. The whole experience of the Logos could not be shared, but 
perhaps a visual representation would excite the full field in the viewer’s mind by 
morphic resonance. : 


The underlying idea was a vibration analogy for mind, brain, and human behavior. At 
present, these ideas have become widespread and familiar, in the form of neural nets, 
excitable media, cellular automata, and complex dynamical systems. But at the time, 
they belonged to the fringe. Further, the edge of the wave of the chaos revolution had 

_ just arrived at the shores of the physical sciences. Some students, still under the 
influence of the lost Sixties, were attracted. Together, we began the macroscope project. 


2. The vibration metaphor. During the European phase of my walkabout, in February of 
1972, I was invited for a short visit at the Institute des Hautes Etudes Scientifiques near 
Paris, the French equivalent of our Institute for Advanced Study. This is the home 
institution of Rene Thom, who invented catastrophe theory in 1966, and David Ruelle, 
who brought chaos theory to the attention of physicists in 1973. Thom showed me Hans 
Jenny’s book, Kymatikf. I was struck immediately with a feeling of urgent importance. I 
called Jenny in Basel to arrange a meeting. Soon I was at Jenny’s home, where he 
showed me slides and films of his work, and shared his ideas on the significance of 
vibrations and Chladni patterns in human physiology. He was a follower of Rudolph 
Steiner. 4 


Later that year in India, Jenny’s ideas were echoed in my readings in the Vedas and the 





abraham MS#47: vismusic 3 7 July 26, 1988 





teachings of the yogis and gurus, especially Neemkaroli Baba. My own experiences, 
repeated during regular inward journeys over the six years, provided further teachings on 
the vibration model for the mind and consciousness. My study of Indian music furthered 
my understanding of the wave metaphor, which is basic to the Samkya philosophy. 


Dynamical systems theory, including catastrophes and chaos, provides a mathematical 
framework for the elaboration of the vibration model. Indeed, this elaboration was 
already underway in the work of Thom. In fact, Christopher Zeeman described to me his 
own explicit model for memory, based on catastrophe theory and excitable media, in 
Amsterdam, shortly before I flew to India. However, it was Jenny’s work, providing 
visual representation of the basic concepts and phenomena, which seemed to me to have 
the greatest potential for furthering our understanding of human consciousness. 


3. The macroscope. With the aid of students from my seminar at UCSC, I reproduced 
Hans Jenny’s kymatic device in my lab. Our device was larger and less precise than 
Jenny’s. We used a four-inch dish for the water/glycerol solution, four-inch telescope 
mirrors loaned by Lick Observatory, and a color schlieren filter developed by Gary 
Settles. An analog electronic tone synthesizer was built especially for the device, and an 
industrial xenon arc lamp provided the illumination. When finished, I alligned the optics 
approximately by eye, turned everything on, and glanced at the screen. I was astonished 
to see a perfect Jenny-style Chladni pattern, in full color. The experience overwhelmed 
me, and I retired to the corridor outside the lab to recover my composure. 


An official opening was planned for the lab, renamed the Jenny Four-Inch Macroscope, 
in July, 1974. On impulse, I asked my Indian music guru, S. D. Batish, to sing at the 
opening. We attached a microphone to the amplifier which vibrates the fluid in the dish, 
in place of the pure tone generator. This event provided my first experience of visual 
music based on Chladni patterns, the essential forms of vibration in three-dimensional 
media. It connected, all at once, my experience with Indian music, Samkya vibration 
theory, Thom’s catastrophe theory, and the light shows of the Sixties. Math, music, 
mysticism, all are one! 


Subsequently, we made systematic use of the instrument (with the tone generator, not my 
master’s voice) to study the bifurcations of chaotic motions of vibrating waves until 
1979. A video is available from Aerial Press describing the instrument and the 
experiments. No other direct reports have been published in scientific journals, but the 
understanding gained from my experience with the Macroscope has illuminated all of my 
writings (especially those collected in my books On Morphodynammics and Complex 
Dynamics). Recently we did revive the macroscope to record a "music video" of Jill 
Purse’s overtone chanting. The routine use of such a device for visual music is 
inconvenient but highly recommended, and this direction has been developed by Gary 
Settles. 
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4. The Visual Math Project. Soon after the beginning of the Macroscope Project, 
computer graphics arrived at UCSC. In 1975, with computer scientist Evan Schaffer, I 
taught a special section of calculus using computer graphics for demonstrations. This 
grew into a state- and federally-funded project to develop a massive teaching system for 
the beginning math courses, called VISMATH. It ended in 1982. 


Part of our VISMATH program was an annual film show, at which almost all computer 
graphics films on mathematics were shown. Each year it was longer, and eventually 
multiple shows were neccessary. As film artist John Whitney and his family, coworkers, 
and followers were using mathematics in their works, we added a number of art films to 
the programs. In this way we became familiar with the visual music medium, as used by 
frame-by-frame film animators. We also got to see Tom Banchoff’s great classic of 
mathematical animation, Hypercube, and Nelson Max’s all-time great Inversion of the 
Sphere, and to meet their authors. In these two films, in particular, I recognized actual 
images from my own inner experiences during my walkabout. Some were so accurate 
that a prior experience in 1969 might almost have been a precognition of sitting in the 
film show in the UCSC auditorium in 1975. Patiently I waited for computer graphic 
hardware to evolve the capability to create and manipulate these images in real-time, so 
that I could use it to share my own inner visual experiences with other people. 

_ Meanwhile, I tried to create suitable images with video feedback. Although these efforts 
failed, they did provided some fundamental concepts relating dynamical systems theory 
(especially bifurcation theory and chaos) to the visual music context. Some of these are 
illustrated in an excellent video made by Jim Crutchfield with an analog video 
synthesizer, available from Aerial Press. But we wanted affordable, programmable 
digital video. | 


5. The MIMI Project. In the early 1980’s, satisfactory digital video hardware became 
available. The Fairlight Computer Video Instrument (CVI) was an early application. 
Soon, a cellular automaton machine (CAM) appeared at scientific conferences. The new 
era for digital video music had arrived. For me, this meant the possibility of replacing the 
_ Macroscope with a digital clone. This could be portable, affordable, and programmable. 
Further, because of the recent evolution of the MIDI standard for the interconnection of 

_ digital musical instruments, a digital video based visual musical instrument could be 
playable, via a keyboard, guitar, flute, violin, or even voice. 


In 1983, people began inviting me to speak at non-mathematical events. My path along | 
the New Age campsites eventually led to a stage in Hollywood, where I proposed a 
design for a Mathematically Illuminated Musical Instrument (MIMI). This is nothing but 
the digital-video based clone of the Macroscope, connected via MIDI to a digital 
keyboard instrument. Sitting at the keyboard, one might sing, play sound, and play 
picture, all at once. The picture part was envisaged as a modulation upon ordinary reality, 
in the form of real-time video. This sort of reality hacking is used in the Fairlight CVI. 
However in MIMI, the operation on a real picture would be defined mathematically, 

- through the algorithms of dynamical systems theory. Among these algorithms are the 
cellular automata invented by Von Neumann to simulate the human brain, cellular 
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dynamata based on the heat and wave equations of mathematical physics, neural nets, 
excitable media, turbulent fluids, and so on. 


For example, with no keys pressed, we might be watching an ordinary video of a dancer 
or surfer. On pressing a key and holding it down, we get a note. Anorgantoneanda_ 
MIMI note. The MIMI note stops the ordinary reality action on the screen, and begins a 
mathematically defined deformation of the freeze-frame image on the screen. In the case 
of an image processing algorithm based on the heat equation, the picture would fade 
slowly to grey. This process would continue as long as the key was held down. The pace 
might depend on key pressure or velocity, pitch wheel, or other MIDI controller. Upon, 
release of the key, the grey would fade back into the ordinary reality of the moving 
surfer. Similarly, there are different ways to interpret a MIMI interval. Two keys pressed 
at once could initiate a combined effect, which may be totally unrelated to the individual 
effects programmed to follow each of the two keys singly. The program attaching a 
dynamic image process to a key (or combination of keys) runs in a personal 
microcomputer, and is available to the performer. Finally, the dynamic image process (a 
mathematical rule followed by the digital video processor) may have parameters, under 

_ control of the performer via breath controllers or other MIDI devices. Thus, chaotic 
bifurcations of vibrating fluids such as seen in the Macroscope may be called up at will 
by the visual music artist in real-time performance. This may provide a new level of 
performance for reality hackers of the future. | 


The first prototype MIMI is now under construction at UCSC, under a contract from 
Pacific Shift, a high-tech studio in Santa Fe, NM. It consists of a MIDI keyboard and 
‘sound synthesizer, a personal computer with MIDI interface, a real-time digital-video 
image processor, and a rack of analog video studio gear. All this has cost about $100,000 — 
so far, but prices are rapidly falling for most of the components. 
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Box 1: THE HISTORY OF VISUAL MUSIC 


Visual music is a traditional art medium, with an extensive (if little known) history. It 
probably played a basic role in the arts and rituals of the cave cultures of the European 
epipaleolithic, and the early civilizations of the Anatolian neolithic. The cave sanctuaries 
and rituals of Minoan Crete, and the derivative mystery schools of Ancient Greece may 
have continued this artistic tradition, along with its mystical religious associations. The 
performances of Father Castel in Paris of color music created with candlelight and cloths 
were famous in the eighteenth century. 


In recent times, visual music reappeared in the Theosophical Revival at the turn of this 
century. Aleksandr Scriabin intended Prometheus: The Poem of Fire (1910) to be 
accompanied by a light show. (By the way, it begins in chaos.) Alexander Wallace 

_ Rimington performed live color music in London at about this time. Claude Bragdon 
and Thomas Wilfred in New York created keyboard instruments for the live performance 
of color music in the 1920’s. In the same period, color music compositions in the form 
of animated films were laboriously made by Arnaldo Ginna and Bruno Corra in Italy, and 
Oskar Fischinger in Germany. In 1936 Fischinger moved to California, where he 
continued his work, influencing John Whitney and others of a major group of color music 
' film artists in America. 


The live performance of color music survived in the work of Mary Hallock Greenewalt 
in Philadelphia in the 1940’s (she invented the rheostat for her organ), Charles Dockum 
in New York and California in the 1950’s, and of course in the Hippie culture of the 
1960’s. 
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Box 2: THE HISTORY OF ACOUSTICS 


The glass harmonica was invented by Benjamin Franklin in Paris in 1761. It was popular 
until about 1800, and many composers made use of it, including Mozart and Beethoven. 


Ernst Chladni was the father of modern acoustics. He was born in Wittenberg, Saxony 
(near Bonn, Germany) in 1756, and earned a degree in jurisprudence at Leipzig in 1782. 
He was an amateur musician, and he designed and constructed two glass harmonicas. In 
connection with this work, he developed the technique (now called kymatics) of 
sprinkling sand or powder on glass plates, and bowing them with a cello bow. The sand 
would move to a filigree of ridges, like the ridges of sand dunes, now called the Chladni 
nodal lines. The regions between the nodal lines, called antinodes, are responsible for 
the sound produced by the vibrating glass plate, its pitch, timbre, and so on. Chladni was 
searching for designs which would have a specified timbre, like the electronic synthesizer 
programmer of today. His first report of this research was published in 1787. Besides his 
acoustic researches persued at home in Wittenberg, Chladni travelled extensively around 
Europe giving performances and demonstrations of music and sand figures. In 1800, he 
arrived in Paris, and made a presentation to the Academy of Science, at the invitation of 


Laplace. Chladni had a private audience with Napolean, which resulted in a prize 


_ competition in 1809 for a mathematical explanation of the nodal lines. Napolean’s prize 
was won in 1815 by Sophie Germain, one of the first great women in mathematics. As 
women were prohibited from attending university classes at that time, she obtained her 
mathematical training by impersonating a truant male student, Lucian Leblanc, at the 
Ecole Polytechnique in Paris. Her prize paper founded the branch of mathematics, now 
known as continuum mechanics, upon which is based the modern science of acoustics 
and the mathematical theory of nonlinear vibration. 


Chladni’s experimental work has been repeated and extended regularly over these two 
centuries, most notably by Mary Waller (London, 1961), and by Hans Jenny (Basel, /76 
(1972). 
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